Elastic monitoring framework for cloud infrastructures

This paper presents a scalable and elastic distributed system for monitoring Cloud infrastructure based on a pure peer-to-peer architecture. Its distributed nature enables to deploy long-living queries across the network to monitor a diverse set of entities and metrics, spanning across all layers of a Cloud stack which can change rapidly. This allows for aggregating low-level metrics from operating systems, to higher level application-specific metrics derived from services, databases, or application log files. The observed metrics and information can be evaluated and used to reliably trigger policies to automate complex management tasks within a Cloud environment. The architecture incorporates a query framework for obtaining high-level information and a policy framework to provide self-management capabilities to monitored Cloud infrastructure. The system has been implemented as a proof of concept. Details and statistical results are provided to validate the scalability of the underlying architecture.